Application of reinforcement learning on medium access control for wireless sensor networks

Yi Chu Ph.D. Thesis Department of Electronics, University of York 2 Abstract This thesis investigates the application of Reinforcement Learning (RL) on Medium Access Control (MAC) for Wireless Sensor Networks (WSNs). RL is applied as an intelligent slot selection strategy to Framed ALOHA, along with analytical and experimental performance evaluation. Informed Receiving (IR) and ping packets are applied to multi-hop WSNs to avoid idle listening and overhearing, thereby further improving the energy efficiency. The low computational complexity and signalling overheads of the ALOHA schemes meet the design requirement of energy constraint WSNs, but suffer collisions from the random access strategy. RL is applied to solve this problem and to achieve perfect scheduling. Results show that the RL scheme achieves over 0.9 Erlangs maximum throughput in single-hop networks. For multi-hop WSNs, IR and ping packets are applied to appropriately switch the relay nodes between active and sleep state, to reserve as much energy as possible while ensuring no information loss. The RL algorithms require certain time to converge to steady state to achieve the optimum performance. The convergence behaviour is investigated in this thesis. A Markov model is proposed to describe a learning process, and the model produces the proof of the convergence of the learning process and the estimated convergence time. The channel performance before convergence is also evaluated.